INFERENCE STAGE

ControlAudio: Tackling Text-Guided, Timing-Indicated and Intelligible Audio Generation via Progressive Diffusion Modeling (2026.acl-long)

Challenge:	Recent advances in Automated Theorem Proving have shown the effectiveness of leveraging a (large) language model that generates tactics (i.e. proof steps) to search through proof states.
Approach:	They propose to use a large language model that generates tactics to search through proof states.
Outcome:	The proposed model solves more unseen theorems with lower trial searches than the current model, which only learns from failed attempts.

Copied to clipboard

Yuxuan Jiang, Zehua Chen, Zeqian Ju, Yusheng Dai, Weibei Dou, Jun Zhu

Speech & Audio in NLP

Challenge:	Recent efforts on text-to-audio generation are exploring fine-grained controllability . however, their performance at scale is limited due to data scarcity .
Approach:	They propose a multi-task learning problem for high-controllability text-to-audio generation . they propose scalable diffusion transformers that augment condition information in sequence .
Outcome:	The proposed method outperforms existing methods on objective and subjective evaluations.

SCV: Light and Effective Multi-Vector Retrieval with Sequence Compressive Vectors (2025.coling-industry)

Copied to clipboard

Cheoneum Park, Seohyeong Jeong, Minsang Kim, KyungTae Lim, Yong-Hun Lee

Information Retrieval

Challenge:	Recent advances in language models (LMs) have driven progress in information retrieval (IR), effectively extracting semantically relevant information.
Approach:	They propose a multi-vector retrieval framework with sequence compression vectors and a coarse-to-fine vector search framework that uses span compressive vectors to score documents.
Outcome:	The proposed approach achieves the fastest latency compared to other state-of-the-art models and can obtain competitive performance on both in-domain and out-of domain benchmark datasets.

Enhancing Cross-lingual Natural Language Inference by Soft Prompting with Multilingual Verbalizer (2023.findings-acl)

Copied to clipboard

Shuang Li, Xuming Hu, Aiwei Liu, Yawen Yang, Fukun Ma, Philip S. Yu, Lijie Wen

Low-Resource NLP

Challenge:	Existing approaches to cross-lingual natural language inference lack annotated parallel corpora.
Approach:	They propose a new prompt learning framework with the Multilingual Verbalizer for XNLI that uses a multilingual verbalizer to align the representations of original and augmented multilingual questions into a unified semantic space with consistency regularization.
Outcome:	The proposed framework outperforms existing methods under few-shot and full-shot cross-lingual transfer settings.

Teaching Probabilistic Logical Reasoning to Transformers (2024.findings-eacl)

Copied to clipboard

Aliakbar Nafar, K. Brent Venable, Parisa Kordjamshidi

Efficient Contextualized Representation: Language Model Pruning for Sequence Labeling (D18-1)

Challenge:	Existing approaches to reasoning using transformers are limiting, resulting in inconsistent results in arithmetic and QA benchmarks.
Approach:	They propose a novel approach that utilizes probabilistic logical rules as constraints in the fine-tuning phase without relying on them in the inference stage.
Outcome:	The proposed approach improves the transformer-based language model’s intrinsic reasoning and makes their probabilistic logical reasoning process more explicit and explainable.

Copied to clipboard

Liyuan Liu, Xiang Ren, Jingbo Shang, Xiaotao Gu, Jian Peng, Jiawei Han

How Certain is Your Transformer? (2021.eacl-main)

Challenge:	Existing efforts to train pre-trained language models have brought significant improvements to various NLP applications.
Approach:	They propose to compress bulky LMs while preserving useful information for a specific task.
Outcome:	The proposed method can detach any layer without affecting others, and stretch shallow and wide LMs to be deep and narrow.

Copied to clipboard

Artem Shelmanov, Evgenii Tsymbalov, Dmitri Puzyrev, Kirill Fedyanin, Alexander Panchenko, Maxim Panov

A Global Past-Future Early Exit Method for Accelerating Inference of Pre-trained Language Models (2021.naacl-main)

Challenge:	Obtaining reliable uncertainty estimations for such neural networks (NNs) is challenging due to the huge number of parameters in these deep learning models.
Approach:	They propose to use Monte Carlo dropout to estimate uncertainty for Transformer-based models and construct inexpensive estimates using Determinantal Point Processes.
Outcome:	The proposed estimates improve the quality of detection of error-prone instances.

Copied to clipboard

Kaiyuan Liao, Yi Zhang, Xuancheng Ren, Qi Su, Xu Sun, Bin He

Empowering Character-level Text Infilling by Eliminating Sub-Tokens (2024.acl-long)

Challenge:	Existing methods to accelerate inference speed of pre-trained language models are limited to local representations of exit layer . current models are associated with large memory requirement and high computational cost, which slow down inference and further encumber the application of PLMs.
Approach:	They propose a method to exit early without passing through all inference layers . they take into consideration all the linguistic information embedded in the past layers a global perspective .
Outcome:	The proposed method outperforms existing methods by a large margin . it uses linguistic information embedded in the past layers and future features . the proposed method is scalable and cost-effective .

Copied to clipboard

Houxing Ren, Mingjie Zhan, Zhongyuan Wu, Hongsheng Li

Challenge:	Existing methods for character-level infilling relied on predicting sub-tokens, but this strategy was ineffective.
Approach:	They propose a method to fill-in-the-mid with Starting and Ending character constraints that avoids predicting sub-tokens in inference.
Outcome:	The proposed method surpasses existing methods and offers significant performance gains.

GenerativeRE: Incorporating a Novel Copy Mechanism and Pretrained Model for Joint Entity and Relation Extraction (2021.findings-emnlp)

Copied to clipboard

Jiarun Cao, Sophia Ananiadou

Enhancing Dual-Encoders with Question and Answer Cross-Embeddings for Answer Retrieval (2021.findings-emnlp)

Challenge:	Existing models for extracting relation triplets suffer from incompletion and disorder problems when they extract multi-token entities from input sentences.
Approach:	They propose a special entity labelling method that fine-tunes the pre-trained model and learns the special entity labels simultaneously.
Outcome:	The proposed model achieves 4.6% and 0.9% improvement over current methods in the NYT24 and NYT29 benchmark datasets.

Copied to clipboard

Yanmeng Wang, Jun Bai, Ye Wang, Jianfei Zhang, Wenge Rong, Zongcheng Ji, Shaojun Wang, Jing Xiao

Right for the Right Reason: Evidence Extraction for Trustworthy Tabular Reasoning (2022.acl-long)

Challenge:	Existing approaches to solve question answering (QA) problems are limited by the need for text generation and answer retrieval.
Approach:	They propose to introduce QA interaction features in scoring function but at the cost of low efficiency in inference stage.
Outcome:	The proposed framework significantly outperforms the state-of-the-art method on multiple answer retrieval datasets.

Copied to clipboard

Vivek Gupta, Shuo Zhang, Alakananda Vempala, Yujie He, Temma Choji, Vivek Srikumar

Context Length Extension via Generalized Extrapolation Scale (2024.findings-acl)

Challenge:	Recent studies show that tabular reasoning models use spurious correlations and focus on false evidence or ignore it altogether.
Approach:	They propose a task where models need to extract evidence and then inference labels . they crowdsource evidence row labels and develop unsupervised evidence extraction strategies .
Outcome:	The proposed approach outperforms baseline models on the inference task using only the automatically extracted evidence as the premise.

Copied to clipboard

Linhan Li, Zhang Huaping

Challenge:	Existing work on extrapolating positional embedding (RoPE) has limited results in the application of long context language models.
Approach:	They propose a set of parameterized extrapolation functions applied to each layer and attention head to adaptively adjust its extrapolations scales.
Outcome:	The proposed model achieves stable extrapolation on 64k contexts by training on 16k length text.

Sparse Low-rank Adaptation of Pre-trained Language Models (2023.emnlp-main)

Copied to clipboard

Ning Ding, Xingtai Lv, Qiaosen Wang, Yulin Chen, Bowen Zhou, Zhiyuan Liu, Maosong Sun

Regularized Contrastive Decoding with Hard Negative Samples for LLM Hallucination Mitigation (2025.findings-emnlp)

Challenge:	Existing methods for fine-tuning pre-trained large language models in a parameter-efficient manner are gaining traction within the research community.
Approach:	They propose a method of low-rank adaptation that enables dynamic adjustments to the intrinsic rank during the adaptation process.
Outcome:	The proposed approach outperforms the current method with a fixed and unalterable intrinsic rank and a low-rank adaptation process.

Copied to clipboard

Haonan Sheng, Dou Hu, Lingwei Wei, Wei Zhou, Songlin Hu

Reasoning

Challenge:	Large language models are prone to generate hallucinations, which can undermine their reliability in high-stakes applications.
Approach:	They propose a method to capture hallucination signals for mitigating hallucis in large language models by regularizing the model's internal signals to a weaker model .
Outcome:	The proposed method achieves better hallucination mitigation performance on four benchmarks.

MetaAlign: Align Large Language Models with Diverse Preferences during Inference Time (2025.findings-naacl)

Copied to clipboard

Mozhi Zhang, Pengyu Wang, Chenkun Tan, Mianqiu Huang, Dong Zhang, Yaqian Zhou, Xipeng Qiu

DiPT: Enhancing LLM Reasoning through Diversified Perspective-Taking (2025.findings-naacl)

Challenge:	Existing methods to align large language models with human preferences often result in a static alignment that cannot account for the diversity of human preferences in practical applications.
Approach:	They propose a method to help large language models dynamically align with various explicit or implicit preferences specified at inference time.
Outcome:	The proposed method can help LLMs dynamically align with various explicit or implicit preferences specified at the inference stage, validating the feasibility of MetaAlign.

Copied to clipboard

Hoang Anh Just, Mahavir Dabas, Lifu Huang, Ming Jin, Ruoxi Jia

Counterfactual Debiasing for Fact Verification (2023.acl-long)

Challenge:	Existing work on improving language model reasoning typically explores a single solution path, which can be prone to errors.
Approach:	They propose a novel approach that incorporates diversified viewpoints into existing reasoning methods to improve their reasoning performance.
Outcome:	The proposed approach can be flexibly integrated into existing models that focus on a single reasoning approach, enhancing their reasoning performance and stability when presented with paraphrased problems.

Copied to clipboard

Weizhi Xu, Qiang Liu, Shu Wu, Liang Wang

Responsible & Trustworthy NLP

Challenge:	Existing methods for debiasing factchecking models learn such biases instead of understanding the semantic relationship between the claim and evidence.
Approach:	They propose a counterfactual framework CLEVER which is augmentation-free and mitigates biases on the inference stage.
Outcome:	The proposed method is augmentation-free and mitigates biases on the inference stage.

A Divide-And-Conquer Approach for Multi-label Multi-hop Relation Detection in Knowledge Base Question Answering (2021.findings-emnlp)

Copied to clipboard

Deyu Zhou, Yanzheng Xiang, Linhai Zhang, Chenchen Ye, Qian-Wen Zhang, Yunbo Cao

Natural Language Interfaces

Challenge:	Existing methods for relation detection only detect one path to obtain the answer without considering other correct paths.
Approach:	They propose a divide-and-conquer approach for multi-label multi-hop relation detection . they propose 'path sampling mechanism' to generate diverse relation paths .
Outcome:	The proposed approach outperforms other competitive approaches on the FreebaseQA benchmark dataset.

Enhancing Recommendation Explanations through User-Centric Refinement (2025.findings-emnlp)

Copied to clipboard

Jingsen Zhang, Zihang Tian, Xueyang Feng, Xu Chen, Chong Chen

Explainability & Interpretability in NLP

Challenge:	Existing explanations for user reviews often fail to meet user-centric aspects, reducing their usefulness to users.
Approach:	They propose a paradigm that refines initial explanations generated by existing models during the inference stage to enhance their quality in multiple aspects.
Outcome:	The proposed model improves explanations generated by existing models during the inference stage to enhance their quality in multiple aspects.

Dual Inference for Improving Language Understanding and Generation (2020.findings-emnlp)

Copied to clipboard

Shang-Yu Su, Yung-Sung Chuang, Yun-Nung Chen

Text Generation

Challenge:	Existing studies have exploited the duality of the task pairs in machine translation and speech recognition.
Approach:	They propose to leverage the duality in the inference stage without retraining whole models.
Outcome:	The proposed method is effective in both NLU and NLG tasks, providing the great potential of practical use.

Mitigating Hallucinations of Large Language Models in Medical Information Extraction via Contrastive Decoding (2024.findings-emnlp)

Copied to clipboard

Derong Xu, Ziheng Zhang, Zhihong Zhu, Zhenxi Lin, Qidong Liu, Xian Wu, Tong Xu, Xiangyu Zhao, Yefeng Zheng, Enhong Chen

Enhancing Court View Generation with Knowledge Injection and Guidance (2024.lrec-main)

Challenge:	Medical Information Extraction (MIE) tasks are a fundamental component of medical NLP.
Approach:	They propose an alternative adaptive constraint strategy to adjust the scale and scope of contrastive tokens.
Outcome:	The proposed approach selectively enhances the identification and classification capabilities while minimizing the influence of other inherent abilities in LLMs.

Copied to clipboard

Ang Li, Yiquan Wu, Yifei Liu, Kun Kuang, Fei Wu, Ming Cai

Class-Adaptive Self-Training for Relation Extraction with Incompletely Annotated Training Data (2023.findings-acl)

Challenge:	Existing methods of natural language generation (NLG) rely on the extensive parameters of pretrained language models (PLMs) but their effectiveness may be compromised by insufficient domain-specific knowledge.
Approach:	They propose a knowledge-injected prompt encoder to incorporate domain knowledge during the training stage, thereby reducing computational overhead.
Outcome:	The proposed approach outperforms established baselines on real-world data in responsivity of claims and in the ability to transfer domain knowledge.

Copied to clipboard

Qingyu Tan, Lu Xu, Lidong Bing, Hwee Tou Ng

FairSteer: Inference Time Debiasing for LLMs with Dynamic Activation Steering (2025.findings-acl)

Challenge:	Existing relation extraction models rely on supervised machine learning, but many datasets are incompletely annotated, causing false negatives and errors during inference stage.
Approach:	They propose a class-adaptive re-sampling self-training framework that favored the pseudo-labels of classes with high precision and low recall scores.
Outcome:	The proposed framework outperforms existing methods on the Re-DocRED and ChemDisgene datasets when the training data are incompletely annotated.

Copied to clipboard

Yichen Li, Zhiting Fan, Ruizhe Chen, Xiaotang Gai, Luqi Gong, Yan Zhang, Zuozhu Liu

Responsible & Trustworthy NLP

Challenge:	Existing prompt-based debiasing methods exhibit instability due to sensitivity to prompt changes . fine-tuning-based techniques incur substantial computational overhead and catastrophic forgetting .
Approach:	They propose a debiasing framework that encodes fairness-related features into separable directions in the hidden activation space.
Outcome:	The proposed framework performs inference-time debiasing without requiring retraining or prompt design . it detects bias signatures in activations and then computes debiased steering vectors . the proposed framework is available to download in the u.s.

Code Prompting Elicits Conditional Reasoning Abilities in Text+Code LLMs (2024.emnlp-main)

Copied to clipboard

Haritz Puerto, Martin Tutek, Somak Aditya, Xiaodan Zhu, Iryna Gurevych

Distillation Enhanced Generative Retrieval (2024.findings-acl)

Challenge:	Recent prompting techniques have improved LLMs’ performance on various reasoning tasks, but there is little understanding of what triggers reasoning abilities in LLM in the inference stage.
Approach:	They propose a method that transforms a natural language problem into code and directly prompts the LLM using the generated code without resorting to external code execution.
Outcome:	The proposed method boosts multiple LLMs by 22.52 percentage points on GPT 3.5, 7.75 on Mixtral, and 16.78 on Mistral.

Copied to clipboard

Yongqi Li, Zhen Zhang, Wenjie Wang, Liqiang Nie, Wenjie Li, Tat-Seng Chua

ReflecTool: Towards Reflection-Aware Tool-Augmented Clinical Agents (2025.acl-long)

Challenge:	Generative retrieval is a promising new paradigm in text retrieval that generates identifier strings of relevant passages as the retrieval target.
Approach:	They propose a framework that leverages generative language models to enhance generative retrieval by distillation.
Outcome:	The proposed framework achieves state-of-the-art performance among the generative retrieval methods.

Copied to clipboard

Yusheng Liao, Shuyang Jiang, Yanfeng Wang, Yu Wang

Gradient-guided Unsupervised Lexically Constrained Text Generation (2020.emnlp-main)

Challenge:	Large Language Models (LLMs) have shown promising potential in the medical domain, assisting with tasks like clinical note generation and patient communication.
Approach:	They propose a framework that excels at utilizing domain-specific tools within two stages.
Outcome:	The proposed framework surpasses the pure LLMs with more than 10 points and the well-established agent-based methods with 3 points.

Copied to clipboard

Lei Sha

Text Generation

Challenge:	Existing methods for lexically constrained generation fail when the search space is too large . a novel method to solve the problem is based on gradient-guided optimization .
Approach:	They propose a method to solve lexically-constrained generation as an unsupervised gradient-guided optimization problem.
Outcome:	The proposed method achieves state-of-the-art compared to previous methods . it is free of parallel data training, flexible to be used in the inference stage of any pre-trained generation model.

Non-Autoregressive Machine Translation as Constrained HMM (2024.findings-acl)

Copied to clipboard

Haoran Li, Zhanming Jie, Wei Lu

Uncovering Main Causalities for Long-tailed Information Extraction (2021.emnlp-main)

Challenge:	Autoregressive (AR) models have some drawbacks due to slow inference speed and label bias due to local normalization.
Approach:	They propose to use a left-to-right Hidden Markov Model (HMM) to control label bias in non-autoregressive translation (NAT) They propose a bi-directional HMM, which can regularize each other's biases via shared parameters.
Outcome:	The proposed models can achieve comparable performance to autoregressive Transformers using various decoding methods.

Copied to clipboard

Guoshun Nan, Jiaqi Zeng, Rui Qiao, Zhijiang Guo, Wei Lu

Boosting Multi-modal Keyphrase Prediction with Dynamic Chain-of-Thought in Vision-Language Models (2025.emnlp-main)

Challenge:	Information Extraction (IE) aims to extract structural information from unstructured texts.
Approach:	They propose a framework that aims to uncover the main causalities behind data in the view of causal inference.
Outcome:	The proposed framework can detect the main causalities behind data in the view of causal inference.

Copied to clipboard

Qihang Ma, Shengyu Li, Jie Tang, Dingkang Yang, null Chenshaodong, Yingyi Zhang, Chao Feng, Ran Jiao

Multimodality

Challenge:	Multi-modal keyphrase prediction (MMKP) aims to produce concise, informative phrases that capture the essence of cross-modal inputs.
Approach:	They propose to use vision-language models to generate conclusive phrases using multiple modalities of input information.
Outcome:	The proposed methods outperform existing methods on absence and unseen scenarios and overestimate model capability due to overlap in training tests.

ShieldHead: Decoding-time Safeguard for Large Language Models (2025.findings-acl)

Copied to clipboard

Zitao Xuan, Xiaofeng Mao, Da Chen, Xin Zhang, Yuhan Dong, Jun Zhou